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ABSTRACT 


A random mathod for generated binary trees is presented, 
and two for its of a class of one person games called '"Tree 
Solitaire"* which have such trees as their game trees are 
defined f After what "look— ahead strategy" means: in terms gf 
such games is discussed, a theorem, on the most efficient use 
of unlimited look-aheads is proved, and a collection of strate¬ 
gies involving 0, 1, or 2 look-aheads per move is introduced- 

A method involving diagrams is presented for calculating 
the probability of winning under the various strategies over 
a restricted class -of games. The superiority of one of the 
1 look-ahead strategies over the other is proved for games of 
the first form on this restricted class* For games of the 
second form in this class, all the introduced strategies have 
their chances of winning calculated, and these results are 
compared among themselves, with the result for the first form 
of the game, and with the results of Monte Carlo estimation of 
the chance of winning in a particular case* 

An approximate method for evaluating strategies from any 
given position is introduced, used to explain seme of the pre¬ 
vious results, and suggest modifications of strategies already 
defined, which are then evaluated by Monte Carlo methods. 

Finally, variants on Tree solitaire are suggested, their 
general implications are discussed, and using the methods 
already developed one of the most suggestive variants is 
studied and the results show a significant reversal from, those 
of the original game, which is explained by the difference in 
the game* on one particular. 
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INTRODUCTION 


In confronting the problem of programmlng computers 
to play games such as chess, where good nlay seems to 
re Quire the ability to 11 look ahead 11 to future positions, one 
is immedlately aware that, desnite the comnutar"c speedy the 
game tree lu much too large to allow looking ahead to the 
end of t".e frame in making the decision on where to move. 

Present moth . It rr,t look to a certain iiir.ited depth in tne 
tree us in " t J on s wh ion attempt to c ■/nl u, ate " how p. ood 

the posit]ona seen arc, end use what they J']nd as best they 
can. Certain techniques such as o-f ha'.e been developed 
to streamline this process, but r,o theoretic framework for 
the problem of look-ahead has been constructed* 

This paper reports an attempt to begin such a 
construction by examining a class of one person cornea with 
simplified game tree, and evaluating various limited and 
unlimited look-ahead strategies for it* In doing so, a 
terminology for dealing with look-ahead is developed, and 
the kinds of distinctions and definitions that umy prove 
necessary or useful for further work are discussed. 

This is an exploratory study with no known predecessors 
in the literature, and so tight organisation and single-minded 
purpose have been avoided in the interest of presenting all 




the varied results so far Obtained from different approaches, 
and ns many as possible of the cart .lectures and suggestions 
for further study thaL have occurred to the author. In 
addition much tlffle has been spent on detailed explanations 
Of problems and methods which, though not Particularly deep,, 
are probably not particularly familiar either + 

Chapter 1 introduces the class of games called 
tree Solitaire (abbreviated TS) and its two forms (TSD and 
TCI), by explaining how the game tree for such a game in 
generated, and then turned into a name. 

Chapter 2 discusses what is meant by look-ah cad 11 
in terns of this class of games* what is meant by "look¬ 
ahead strategy" j ar.d how such are to be evaluated. It then 
proves a few simple results and defines the look-ahead 
strategies that will be dealt with in later chapters; STHAT 
and NG3TRAT involving no look-ahead* HSTRAT and LETRAT 
involving one, and BSTRAT* 2HSTRAT* and 2i*STRAT Involving 
two look-aheads. 

In chapter 3 methods are introduced which produce 
closed formulas for the probability Of Winning using HSTRAT 
and L5TRAT in games of form TSD # It Is proved that 
LSTRAT is always the better strategy In such games. 

■Chapter 4 uses methods similar to those of chapter 
3 to evaluate* by means of computer program, all the look-ahead 
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strategies defined in chapter 2 t over the class of ftaines of 
form TSI, (Alternative evaluation by Monte Carlo methods 
is discussed for HSTRAT and LSTRAT*) The strategies are 
then compared as to their “effectiveness 1 ' in obtaining wins 
when -ar-ious Parameters involved in the generation of the 
ft ames are varied . 

Chapter 5 introduces the concept of '’depreciation 
factors' for the purpose of evaluating strategies from 
positions other than the first one in the ftame 4 ft Dlcusible 
argument is ftiven for the predominance of L ST RAT o.'er 
RfTP.AT In TSI. Modifications of H5TRAT and BSTRAT are 
suggested, usinft these depreciation factors, and evaluated 
by Monte Carlo metnods so that they cam be compared to the 
oriftinal s t rat efties. 

Chapter 6 introduces variants on Tree Solitaire 
as a means of sucftestinjT ways of extending results a ready 
obtained. In particular a game which seems a much more 
reasonable idealization of ftames like chess is introduced, 
and a start Is made toward applying the methods and results 
of the previous chapters to it. 
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CHAPTiiH 1 


TREE SOI.ITAIRK AND ITS GAME TREK 

Most r.Me playing programs using lock-ahead abstract 
a gome tree From the rules of the gams* and assign values to 
the noinis in the tree according, tc an evaluation Function 
defined on the physical eonfigurations of the corresponding 
nosit^ons In the “shr. In studying tne workings 0 J1 the 
? oor.-shead uroced-re theoretically* one is most interested in 
luct the game tree itself and the values assigned to its 
points* Therefore ih this report the procedure is to generate 
the game tree* aisLgn values to its points* and then to define, 
with as much simplicity as possible* a game which tne tree 
represents, 

Most consideration is given to a one-person game* or 
actually class of one-person games of this type* which will be 
called "Tree Solitaire" (TS}, and has been developed from an 
idea proposed by Seymour Papert.^ Variants on this name will 
be discussed in Chapter 6. Trees for T.H are binary* end 
generated as follows - 

Start with an initial "fork" as shown In Figure 1,1 


1* In unpub1isned notes and conversation. 
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The top point of tie form will be called a "split point' 3 . 

Top or '•'Split" pdjnt 



Figure l . 1 j a fork 

The bottom points oi the fork are called the ironed late 
successors" of the too point, rind aonetines "branches, v 
The bottom points of the initial font may be either further 
"split points" or simply end points, Whether a bottom point 
is a split point or an endpoint is decided by some probabilist 
method, chosen so that the probability that the point is an 
endpoint is a f^iven number E* The probability that it is a 
split point is S = 1-E. Whenever no confusion will arise, 
an endpoint will be referred to as an E # a split point as an 
o. If a bottom point to a fork is determined to be an z, a 
new fork is added to the tree with this point as its top point 
and two new bottom points must be deai t with. As soon do all 
available bottom points have been determined to be E's T the 


- t 


generation c f - e. binary tree is complete. Figure 1.2 .alves 
an example. 



Fin.ure I.;: A mnerated tree 

The tree will be r vencrated wit:- a finite number of 
points if E > .5- 1 The need for this will occasionally be 
apparent p but many of the results to be obtained will be 
valid for E < , 5r and this will be explained when It occurs. 
Assign to the top point of the tree the value 1, In 
some probabilistic way, decide Oh two number whose sum is 
this top value, arid a&eiftn these two numbers to be the values 
of the branches of the fork. Thus in Figure l.lj the ton 
point has value 1 and the bottom points mishit have ’-slues ,6 


1. Harris, The Theory of Br a nch ing Processes > p*7 
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&nd ■ ,4, For any Fork* the ratio of the value of the higher 
valued branch to the top value is- called the "split ratio". 
The split ratio here would be .6. IF a bottom point is an.. 
S, we split Its value between its two branches in the sane 
probabilistic Way. This process is continued until nil 
points oF the tree have been assigned value®. In a Finite 
tree the sun of the end points will equal 1. 

In order to construct a g,vme K some of these end¬ 
points will bo c-'jl: led wins, the rest ,osaer:* There will be 
no drew?. Two si;r.li r:r -tot nods of assir:r<ln.v wins and losses 
.ti ,'fi rise to two different Forms oi' Tree Solitaire, 

The dependent Form (TSD) treats the values of the 
endpoints as a probability distribution, ana assigns one w5n 
to the F^ame, locating It at an endpoint chosen according to 
the distribution. All other endpoints rare losses. Thus the 
value of an endpoint is the probability that it is a win. 

The reason that this Form of the ^ame is called dependent is 
thati if one endpoint of value A is known to be a loss, 
then the probability that any other endpoint is a win is its 
original value divided by (1-A)* since we know that one of 
the endpoints Rust be a win. 

In the independent Form of the game (TSI)j the /al.ue 
of an endpoint is used as the probability that the endpoint 
Is e. win * the decision be inf; made in de p enden 11 y for each 





endpoint. Thus there need pot bo exactly one win. There 
may be zero, one, two, three, etc. Knowing that one endpoint 
is a loss gives no information as to the others* This form 
of the Raffle was introduced because it seemo closer to rea] 
"E.7ies than I'-IE, and a' so because it was erroneously thourht 
that T.ID would be too complicated to analyse, (^ee Ch-mter 


To complete the definition of Tree Solitaire, one 
must explain trie rules of play* Clearly, ithe Jessie tree 
in to worst at FAfl* trees shoulti, a "play' 1 of the rane "rill be 

representable as a path from the top point to an end point., 

with the points of the path representing positions in the J^ejiie, 
and the lines between them representing' legal no-es* Tnio is 
exactly what is done. 

The r positj on !l corresponding to each point is a certain 
set of information about that part of the tree: 

1. The value of the point itself 

2 . Whether it is an E or an C 

3. If an £, whether it is a win or loss 

4* If an S, the values of its two branches, or 

equivalently, its Split ratio, 

A move is made from a position corresponding to ah $ 
point to a position corresponding to one of its branches. A 
binary choice is involved and herein lies the opportunity for 


n 


strategy. The reme starts In the position corresponding to 
the ton point of the tree and continues until the position 
correspendin^ to an endpoint is reached, ond is won IT and 
only if that point is a win* From now on "the position 
corresponding to" will he omitted and positions wild be 
identified with the corresponding points * hopefully without 
loss of' clarity. 

It would be apropos nere to exnlaln how this game 
compares to more comp.tented ones 11.pc chess, and to .Justify 
Its ust ns a starting point in the study or the problem of 
loo?:-ahead. Fl*"St of all, the simplification to a one-person 
game is reasonable in the beginning of such, a study, -aft well 
, as convenient due to trie nature of the game tree and its 
valuation. Similarly, having the tree be binary and thus 
having only two possible moves from any position, seems a 
reasonable simplification, as does the fact that an ordinary 
’’play" of the game will only comprise a small number of moves. 
Letting the va ; ue of a position be its probability of being a 
Win is e straightforward valuatJon, even If It is perhaps more 
powerful and simpler than those used in chess playing programs. 
In TE the chance of any position being a final one is a 
constant E throughout the game and this may give rise to 
doubts, but is perhaps ,justifiable by the fact that in games 
like chess a number of the moves from any position are such 
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outright blunders as would create a lost cattle for the mover -- 
Eind a win for ills opponent* 

The principle objection to T,^ must arise from the 
fact that the values of positions of necessity decrease as the 
play of the j;ame procedes farther down the tree. This is- a 
reasonable criticism, and the weakness rei'erred to does have 
an appreciable effect on the results, as will be seen When 
they are cemnared with those for a variant class of pontes 
without this weakness , Sn Chapter 6* 

Hfiwever* Tree -jOlitaj.ro is a nice showcase for toe 
■ arious possible limited look-ancad strategies* and allows 
much straightforward analysis of them. It can serve as a 
workshop for developing nethods of evaluating 3 tro.te~.ie 5 , and 
it as well has a certain amount of intellectual appeal all by 


itself, 


CHAPTER 2 


LOOK-AHEAD STRATEGIES AND THEIR EVALUATION 

In the content of Tree Solitaire, a ' look-ahead" will 
be defined as seeing a suncescor position to the one you are 
in 3 or nave r-.l ready Men, by "seeing a position" will he 
meant the acquisition of the entire set of information asso- 
C-inted with the loosition, as defined in Chapter 1. The 
difference between a "look.-ahead" and a ''move'’ is that lockin': 
ahead to a. position, although it gives you n.z much information, 
as moving there would, does not commit ycu to that position, 
whereas moving there would 

A look-ahead strategy will work in the foil owing way* 
'iupposc you find yourself in position X, In accord itfith a 
given rule you choose those succeeding positions at which you 
will look -- the rule may allow you to use the result of your 
earlier look-aheads in choosing where to use your later looks. 
Once the look-aheads have been completed, you make use of the 
information to decide according to a second rule where to move, 
thus arriving at a new position, where, if it is not the end 
of the game, the process is repeated* Any information you may 
have gained in previous look-aheads as to the successors of 
this new position is as&umed to be forgotten. 


It 


A look-ahead strategy can thus be seen to bo defined 
by giving the two rules that govern it: the one that tells 


where to look,■ and the one that tells where to move* given 
the results of the look-ahead* 


Strategies may be evaluated according to a number of 


criteria. High on the list* however* must be their effec¬ 
tiveness at achieving a win* over the whole class of games of 
Tree Solitaire* The principles of such evaluation are 
demonstrated in the following two cases of strategies 
invo 1 v in r; no loo k- ahead at al 1 * 


HOSTRAT is defined by the rule that moves are chosen 


completely randomly* without any reference to the values of 
positions* In this case the expected value of the-position 
you move to will be 1/? the value of the position you are in 
The chance of winning on a given move is the expected value of 
the position at the end of that move times the chance of 
/rettine; that far in the game, under the given strategy! times 
the chance of the position being an endpoint* i^e.j times E, 
The chance of winning the game at all thus becomes a conver¬ 
gent infinite sum* In this case* 




v 


E/(0-3) * *= E/(£+l) 
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If E --- * then the nrobability of winning W. r „ = . RR13- 

HOST RAT is not the best possible strate^ Involving 
no 1 ock-ahoeds* That honor belongs to the obvious strategy 
of j=l1v/ ay 3 moving to the higher valued of the two branches of 
a fork* which straterry will be called simniy ETRAT. To 
calculate tne chance of winning under STn.'-T, we make use of 
the expected va'ue A of the split rati* as a prediction 
for the way values split at every split point we encounter* 
If the top point of a fork nec value X„ the expected valve 
of the high branch's value is AX and for the lower - slued 
branch BX> where B ?* (1-A). Since the value at the top of 

the tree is l a the cnanoe of winning under . ETRAT can be 
seen t* he 


*[_ AS{A5) M 
n=o 


_A£_ 
T - AS 


AS AE 

1-A+AE ~ S+AE 


1 a 

If E = , A ■= ]j t then, the probability of winning 

W = , 6000 . 

Against L| effectiveness 1 ' must be weighed the '’cost 11 of 
the strategy,, in terms of, say, computer time. Since it may 
be assumed in tne case cif look-ahead • strategies that the 
principle time consuming factor would be the number of look¬ 
ahead a, cost considerations would lead us to want to minimize 
this number per move over the class of all game trees. 


16 - 





However, In trying to bal anc e this against the importance 
of having an effective strategy, any theoretical treatment 
would involve a certain amount of nrbitrarlness* which mir.ht 
have a considerable effect on the results. In order to avoid 
this problem, this study nas concentrated on comparing as to 
their effeetivenes e various strategies whose costs Jtre the 
s-sme. "Limited loot-ahead" strategies involving one loox- 
a.he?-d w 5 11 first bn examined, and then those Involving two 
find cos sib J y ciore, 

H 

tiut first, for tne sake of perspective, we w*ll note 
a result at the other end of the spectrum -- unlimited look- 
ahead, With unlimited look-ahead a win is assured ^in TAD; 
in TSI a win is assured if there in fact _5_s one). So vnat 
we must do here is pick from a number of equally "effective" 1 
strategiesj that which is least costly, Such strategies have 
irTTonMon the fact that they use their look-ahead to find the 
endpoint which is a win (in TSD; in TSI, such an endpoint 
if it exists), and than the proper choice of moves Is a matter 
of course, 

Intuitively t the test of these look-ahead strategies 
would be the one which looks at the highest valued of the 
points open, to look-ahead i.e fc , the points which axe successors 
to the points already seen. Thus in Figure £,1 you would first 
look at ,6, then H then ,^, .4j .3, .1, 
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Figure ^1: A cample tree with sample vf-tluas. 

Papert 3 proved that this Is Indeed the test method. 

Proof; First note that since any method of selecting what 
point to look at nejtt is independent of whether that point is 
an E or not ? over the class of all f^ame trees the proportion 
of E's looked at to the total number Qf points looked &t 
should he the same * that is, should be the number E. Thus 
to minimize the total number of positions looked at, over the 
class of all ^ane trees, one need only minimize the number Of 
endpoints ]coked at + 

1* In unpublished notes. 
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The expected, number of endpoints to bo looked at in 
order to find a win is* for any strategy in games of TEI; 

A l l + (l-A 1 JA ? -3 + t , . (l" A rj _i^ J ’ n ‘ r - 

and for ToD 

A 1 _-l + ^ ? r ' J + — ■ + A n *n 

where is the value of the i^* 1 endpoint looked at, and 

n is the number of endpoints. The ^1-A^) are cancelled in 
the formula for T£D because, if A^ is not o win> then the 
chance that A.^_, is a win is increased to A^ + (l-A^). 

In both eases, the sun will be minimized if the end¬ 
points are ordered so that > > ••* > which cem only 

be guaranteed by the strategy under consideration* Q.E.D. 

We now return to strategies involving a fixed number 
of look-aheads. The present chapter will conclude with a 
description of the strategies to be evaluated in the next two 
^chapters. However, first it will be noted that unlike the 
strategy .just described, they are all what will be called 
'■ ordinal" strategies. Ah ordinal strategy is one which does 
not make use of the precise values of the two branches of a 
fork, but only notes which one is larger H Only such ''ordinal 1 ' 
information' is used in deciding where to look next, as well as 
choosing where to move next* The use of the precise values. 


- 19 - 





either to choose the next move, or to choose where to loox 
ne?ct t at first thought would seem quite valuable, however 
- the actual Advantage in the case of- one or two look-aheads 
mey he minutcj and there is a correspondirtj\ increase in the 
amount of cal saint ion needed and hence time used. These r.ji-i 
otrwr quest ions .ttq *U sc ussed in chapter 5- 
The a tr ate tie s = 

E:,“i Mui. >'C -■•r.c'.'.'J* Look nt tho higher IJ' it 

i r. an -i s move there. li it is Jin E and a Win* move there. 
Otherwise move to the lower branch, 

LSTRAT. One look-ahead. Look at the lower branch, If it is 
an L find a win, move tncre. Otherwise move to the hir.hcr 
branch. 

ShiSTrtAT. Two look-aheads ( or lcss)„ Look at the hi'rth branch, 
If it is an 5* look at its hir*h branch.,'' IT either of these 
two is a win, or if both arc S’s, move to the high branch, 
Otherwise, move to the low branch. 

dlLSTRAT* Two look-aheads for less). Look at the low branch. 
If it is an 3, look at its hi^h branch. If either of these 
two is a win, move to the low branch. Otherwise move to hif*h 


branch. 



EsSTKAT. TWO 1 ook-aheads* Look at botn branches. If eitner 
Is a win,, move there. Otherwise, wove to the highest ol the 

-two which is-an 3. If both are losses,.accept defeat with 

a smile, 

These five exhaust the possibilities for ono and two 
3o ok- ahead ordinal strategies, in the sense that any other 
such strategy can be Shown to be Obviously inferior to at 
]east one of these which agrees with it as to number of look- 
are ads, whereas the ranging of these is not at first obvious. 
The next two chanters try to- discover this ranging.. Although 
the results in the present chapter ho3d equally well for both 
T3TJ and TS1, the treatment is different when look-aheads are 
involved, end so one chanter is devoted to each type of i^ome 


separately. 


CHAPTER 3 

ONE LOOK-AHEAD STRATEGIES IN THE DEPENDENT FORM 
OF TREE SOLITAIRE 

The results in this and the following chapter will 
be Droved for the restricted class of r^ane trees With 
constant split ratio A, +5 ^ A ^ 1^0. It is a elans ibis 
conjecture tnat the results are a pnroK i mately true over the 
c I -is s of all r ^sltic- treeSi so lonrr as A is, the expected 
'.i 1 a] us -of the split rati o. 

In tnis chapter it will be proved that in TGD* 

1ST HAT is more effective than, HSTRA.T for 3.11 allowable 
values of A and K. Is this a surprising result? With 
unlimited look-ahead the least costly strategy that insures 
a, win involves looking at the highest valued position firstj 
with one look-ahead the most effective strategy is to look 
at the lowest, But this is no contradiction, Also-, we must 
remember the whole strategy* not .iust the look-ahead rule, 
LSTRAT, by looking at the smaller branch* insures that if 
there is a win at either branch on this move, you will ret it. 
If this is the virtue of beinf; ’'ai^ressive 11 , then HSTftAT 
has the advantage of ’"cautious, 11 for it assures you of not 
losing on the current move unless both, choices are losses. 

It is difficult to see which factor outweighs the other, so 
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wa turn now to the actual evaluation of Lira strategies< 

Wow that look-aheads have been invoked, it seems 
unlikely that the probability of winning under a f^Iven 
strategy will be calculable in as simple a manner as was 
used in chapter h. The use of a single exouctnd value for 
the value of i.he nositi.cn on ti "Iven move fat a r^iven ply of 
the f.^rune tree] will lead into difficulty. With look-ahead 
Straterrj.esj the probability distribution of position vn’ueo 
at a /riven ply is dependent on the probability iistrlbution 
for the previous ply. This means that the expected value 
for the position value at one ply is not a direct function 
Of' the expected value fer the previous ply? but of the entire 
distribution of position values. Thus we cannot, as was done 
for iTRATj have a single estimate of the position value at 
one n1y± and use that to calculate our estimate for the next. 
Instead wg must find a way to retain distribution in formation. 

For ^arnes with a constant split ratio this is not too 
hard* and hopefully the probabilities of wj lining thus obtained 
will be /^ood approximations to the probabilities for the class 
of all pannes whose expected split ratio is equal to that 
constant, 

With constant split ratio,, the number of possible post- 
tion values at the n th ply (with the top point of tha ^ame 
tree beinK ply 1} is precisely n. For example, with st> 1 it 



ratio A and E - 1-A, the oossible position value -it tne 

first ply is 1, at the second ply the values axe A find B, 

r , 

at the third A'* AB, and B , and so on. To svaluato the 
chance of winning for a fivers E and A under a particular 
strategy we sun.! for each t)]y f the product of each possible 
position value times the probability that such v position is 
an endooint tidies the probability* over the class of a 1 ! game 
trees with split ratio A and ending nrob ability E* that 
you could get to such a position at the ;-,i ven sly icier the 
strategy. Than we sum tne values thus obtained fer the 
probabilities of winning at each ply over all plies from 
P'iy on. (This is the same as summing the chances of winning 
on the first move, second move, etc + J The tot a'- is tne 
probability ol" winning at all, under the given strater 1 ^ over 
the class of all games with the particular split ratio and 
ending probability. The sum will converge, as it is clearly 
bounded by 1. 

To aid in this calculation* it is convenient to have 
a diagram as in Figure 3+1 for HSTRAT, The diagram is only 
partially dram here. However, it is in a sense recursive, 
and once one understands how it is generated, one does not need 
to see the rest. As such diagrams are much relied upon in 
this report* a detailed explanation of this one will be given 


here, so that the reader can generalise end thus understand 
later diagrams without much further explaining 



Figure 3.1t Evaluation Diagram for HSTRAT 

The points and lines have numbers associated, with 
them* The points represent possible positions you (Hey bs in 
or looking at, and the circled numbers are their values. 
Lines are directed from left to right or from top down, and 








the numbers associated with them are the probabilities or 
rroln^ from, the starting point to the endpoint (in the ease 
of- H31RAT, -of-^ointr from 1 oo kirtn; at. the . .hirrh. branch to 
mo”inp; to the Iovj branch, or of go inn from the too point of 


a fork to loo king; at its high branch). The first horizontal 
row of points represents the first move (or Second ply), *ndi 
so on* The probability of being in a given position {i.e. * 
at a given noint of the diagram) on a. given move (ic, , at a 
ri^en nly of the game tree) Is calculated recursively by 
associating to ea.cn noint in the diagram a, second number [its 
probability") eoual to the product of this number for the 
point's immediate predecessor* with the number On the line 
connecting the two, {In some later diagrams a point may have 
more than one such predecessor, in which case we take the sum 
of such products*) The first point in the tree has 5.00 for 
its probability. 

This particular diagram is rather simple. For instance, 
on the first move the higher branch baa value A. You move 
their unless It is a loss, the chance of which'Is E(l-A). 

If It is & loss, all other probabilities have to be divided 
by (1-Aj because we're in the dependent form of the game,, 
and so the lower branch, to which you move, now hsa value 
{^} - 1. ®4 is so labeled. Essentially this safne kind of 
process goes on at each move* 







Now Tor the evaluation itself. The chance of winning 
on.the first move is 

EA ■+ iSfl-Aj-E 1 ! - Ar' + Br"' 

vxhere o (1-A). On the second move it in 


4 - S(W, jE 


R 


o-n 


f (]-A)E r,A + (l-Aj'E J S 


(remember -■ = 1-E; , end so on, Howe 'or, cancelling tine 
(1 - A ‘) and ni mp ■ Ifyln-;, toe second term is 


(SA + BES) (/'£ f BE y ) 

or (isA + BEE) times the chance of winning on the first move. 
This turns out to be true in general* i, e« * study of the diagram 
and the uey the numbers in it are as.signed shows that 


LtM-MA: Tne chance of winning on any given move (except the 
first) Is (SA + BES) times the chance of winning on the 
previous move 

This means we have a geometric series as before d and so we 
have proved 

THEOREM 3.1: Over the class of all games of TSD with 
constant split ratio A and ending probability Ej the 
probability of winning under HSTRaT is 
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k(m lii-:j 

v{ d l-'ifXibtO 


vjiicrc 


B - 1 — ft, t i r J m 1 "* ^ * 


For example, if 8 - 1 / 2 . A - 3 /*. then » H “ 7/9 * ' TrrH ‘' 
Similar i/cod things happen Tor L 3 THAT In T^D. The 
digram is Figure where the possible positions at each 

ply are located in narrow bands of not quite colinear points. 



Figure 3 . 2 : Evaluation diagram for LSTRAT 


- - 





This, diagram is explained as follows. The first thin?- you 
do Is look at the 1 ower /alued branch of the initial fork, 
Whose.value, is. B. 1-'. it is a win, you move there. If it ip 
a sol It Point, which happens with probability £, you move 
to the higher 'eJued branch, whose value is A. If it is a 
loss (probability E(l.-I5)) ? you still move to the hi^Jior 
branch, but now the probabilities of winning are the ori-'Jnal 
values divided by (l-B) so the value of the higher branch 
is /■ '(1 - B} = ■ i. In a similar way the rest of the 

1 ia.tra;:: may be generated* 

The chance of winning on the firsr, move is 

EE ■+ SEA + E(l-B]E*1 = EE + ESA + E*A 

Oh the next move, the chance of winning is 

E3- h A 2 * ES^AB t E 3 S 3 A 3 + ES 3 A 3 + E 3 £AB + E 3 £/ 3 
or 

{S’A + ESA) (EE + ESA +■ E £ A) 

The reader may verify for himself that a^ain we have 
a ratio which holds from move to move throughout the 3;ame» 

The sum of the resultinc geometric series is 

EB+ESA+E 2 A E(BJ-SA*(1-S)A) e 

" 1 : U-E)SA-ESA 1-SA 





arid we nave shown 


THEOREM 'j.,?: Over tne class of all ryvnes nr TfD wJ th 
constant split"' ratio 'A arid ending Tsaroloability E, the 
p robability of winning under lstrat is 

v; L ^ S-'( 1 -GA] 

where S = I-B t 

if ii - i,-r? and a ; - jj, then ’j. r L - V5 --- .^w. 

[low we art at Last Prepared to nro'.-e 


THEOREM 3.3: Over the class of all :*euraes of TOD with 
constant split ratio A and ending probability E, for any 
combination of values of E and A* .5 < L ^ 1. ‘ and 
. 5 5_ ^ < 1 ’ * both LSTRAT and HSTRAT are more effective 
than STRAT [no looK ahead), and LSTRAT is more effective 
than HSTRAT. 

Proofi This result is Obtained by simply comparing 


EA 

1-SA 


(for STRAT } 


u - E (AfBTC> 

II l-STA+BEj 


and 



The first is clearly smaller then either of the second two* 
so all that need be shown is that W L > * which will be 
true if 


E E U+I51 
I^Ia “ T^S(A+BE j 


> 0 
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That will be true It' 


i ( c ( , 

if 

1 - S(AtBE) ■ 

1 - SA - SBE 

1 . e., 

if 

1 - SA * SBt 

i,c. , 

if 

1 - 5(A+B) +■ 

i, g. . 

*- 

i - rs + 

i. n. t 

.■: *■ 





Bub thin last is obviously true for , 5 < R *■ 1. and 
* t> < Ac 1* Hence VI ^ > *f H 
Strategy* E, D t 


S) > 0 
ab > o 


and ' LSTRAT is the more giTectlve 


CHAPTER 4 


STRATEGICS IN THE INDEPENDENT FORM OF TREE SOLITAIRE 


T:U1 


plate 


The neatness of the formulas for and 

is due to an inherent cancellation which does not 
there for tne two-loch-ahead strategies, so the 


in 


tale 


treat¬ 


ment Of’ these has. been postponed to this chapter on T£I< 

In the case of this class of frames, no closed formula has 
yet been discovered for the probability of winning under any 


of the strategies involving look-shead. 

However, diagrams such as thoro in the preceding 
Chanter can still be constructed* and suggest the ideas for 
recursive computer programs which can calculate the appro- 
priate probabilities of winning (W^, and by analogy, 

W £H * W ?L * and V; B ) move by move* This process, it turns out* 
peed only be carried on for the first ten moves to get 5-plaee 
accuracy* although occasionally a small correction! factor must 
be added to take care of the chance of winning after the 10^ 
move, 

A separate program has been written in MAD for each 
of the five strategies* in addition to one for two limiting 
cases which will be discussed below. Each program when given 
v&luos for E and A outputs a sequence D(l), D(2),,.-,D(10) 


of numbers representing the probabilities of winning on tne 
1st, 2nd, etc. move* An output program then prints them and 
the ratio between succeeding terms* as well as their sum and 
any correction factor* The diagrams and programs are 
presented in Appendix A* 'With the exception of SH3THAT 
and ^LSTSaT, all these programs were checked against hand 
calculations. Those two were too complicated lor hand 
calculation* but the internal workings of both programs were 
t ho r ou r.h l y r on e o v er an d ft u nd satis:; a a t or y, 

In the Case or the One look-ahead ::traegieS* the 


conclusions of the ^receding chanter teem 

to carry 

ove r to 

TCI* even if their proof 

does not. Table 

1 

give 

the palcu- 

lated values for W^j W„ 

(ar.d W) for various 

combinations ot 

E and A* 





As can be seen. 

W L > « H for all 

E 

and 

A given 

here, and both W 1 ^ and 

irf are greater 

11 

than 

VJ. 

For fixed 


E, k? H {and indeed W) approach from below as A 

approaches 1, In the limiting case they would all be equal* 
as all the value of the top point of a fork would go to its 
high branch. Similarly, for fixed A, vl R (but not tf) 
approaches from below as E approaches 1* again viith 

equality for the limiting case, because then both bottom 
points of tne initial fork would be endpoints. Doth these 
conclusions could have been made for T3D* because the numerator 
of the fraction representing - W H was 


Table 4 


E 

A 

w 

W L 

W H 

.5 

. 5 

.33333 

. 55159 

. 4-8110 

.5 

♦ 6666 ? 

.5 

. 63029 

. 590 ; £ 

.5 

.75 

. 6 

. 68768 

, vol09 

.5 

.9 

’ 6 l 8 l 8 

♦83914 

.03268 

c, 

.'>5 

.90470 

.91095 

. 1 -0899 

r 

■i 

, ■'/- -2. 

r,a r V , 


cr 

■ 

♦ 999 

. 998vv 

■ jj ^ 

. 9981 , . 


.5 

.5 

.33333 

.55152 

.4B11Q 

,75 

.5 

. 42857 

. 666 9c 

.64265 

,9 

♦ 5 

.47368 

.72041 

,71253 

GO 

"■ _ V 

♦ 5 

. 4$ff49 

.74717 

♦74653 

.85 

.75 

,42789 

.3184? 

.46770 

■ 5 

.75 

► 6 

.6S768 

.66102 

♦ 75 

.75 

.69831 

,76722 

.75745 

Q 

■ -I* 

.75 

.72973 

.7969& 

♦ 79371 


. 1+ Commiter remits: evaluation of probability 
of winning, under STRaT. LSTRaT, and H3TRAT 
over the class of games of TSI with constant 
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split ratio 


The case in the table with E < .5 can be included 
because in TSiC a «“Sune can be played without generating the 
entire pome tree, All that need be generated are the positions 
tnat will actually be looked at or mo . r ed to under the riven 
strategy* (This is how computer simulation of the game nro- 
cedcs in the Monte Carlo method of evaluation to be discussed 
below.j The expected length of a path in the tree is only 

1. 1;], so by f;uch ’ Lmited generation we nan reduce our work to 
reasonable amount e\:; long c-.s '] is not too sms]].. 

actually ? two factors disturb this estimate of rams 
len:;th when we ere den ling with strate _ ies involving look-ahead. 
Endpoints which are losses may be foreseen and avoided, On the 
other hand* endpoints which are wins may bo foreseen on : 
purposely sought. The results of actual simulation of LSTRAT 
with E ^ . 5 and E e show that playing a given amount 

of frames for the smaller E takes only about 1,4- times as 
long as for the larger E. The factor for B5TEAT is about 

2 . ^3 reflecting perhaps the greater ability of that two-look- 
ahead strategy to avoid losing endpoints. 

Since computer simulation of game playing has been 
introduced], it would perhaps be appropriate here tc present 
the results of an attempt to estimate and in a 

special 'ease, by Monte Carlo methods, E was set to , 5, A 
to ,75* Actual, games were played under LSTRAT and HSTR/iT, 
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with the game' tree being generated as needed with the aid of 
a MAD random number generator. 20j000 or more games wore 
clayed to obtain the estimates, A "trial" was a series oi 
10,000 games. The variation between trials for any particular 
estimate was at most ,000^0, indicating tonsidsr-nble 
stability, The results are summarised in Table 4.2. 



Diagram Method 

Constant Gp3.lt Ratio 

I II 

Variabl c 

I 

S-ol it Ratio 

11 

■ J T, 

»6ft76ft 

.67990 

, fry/y 

.69^0 

.69910 

W H 

,66102 

,659iO 

,66640 

.67070 

,65930 

Table 

4,2 Monte Carlo 

A - ■ 75- 

estimates of W L and 

Wjt for 

E = .5, 


This table, ih 

addition to comparing 

the Monte 

Carlo 


estimates for constant split ratio with the value obtained by 
diagram method in Table 4.1* gives the estimates for W L and 

when the split ratio is not constant, but obeys a probability 
distribution such that every value between .5 and 1,0 is 
euually possible (and hence the expected value of the split 
ratio is ,75), Two different value (I and II) for each 
split ratio alternative are given, because it was found that 
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different utilizations of the random number generator Rave 
rise to significantly different estimates* Under utilization 
I, low ’'allies of the random number caused wins* Under 
utilization II t hl.^h values caused wins. 

The difference caused by this effect is not the eerie 
for both strategies:, or for both split ratio Possibilities. 

Under constant split ratio, the difference for W T is *01100* 

■Li 

for vf^ .00710* Under variable split ratio* the difference 
for irt is *00^70, for w .011*10, /ill these are eonsi- 

I ■ Cl 

derably greater than the .OOC4 g margin of error in the 
individual table entries* thus indicating that these differences 
are significants and „ although the table Jj . 1 values do lie 
between the I and II values for constant split ratio under 
both strategies, casting a doubt on the use of Monte Carlo 
methods for precise estimation of probabilities of winning, 
at least with the present random number generator*^ However, 
such methods do, at least here, preserve the ranking of the 
strategies, and it can be noted that apparently, the Riven 
non-constant distribution of split ratios here does yield a 
greater probability of winning under both strategies than for 
constant split ratio, even though the expected value for the 
ratio in each case is the same. An example of a Monte Carlo 


1, Time pressures have prevented the devising of a random number 
generator which would be for our purposes less biased. 
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program is listed in Appendix B. 

He turning to the diagram method of evaluating 
strategies, and the assumption of constant so] it ratio,, it 
is interest inn; to note now the chances of winning on p arti¬ 
cular no.'-5:3 (i.e., at narticular ply in the .frame tree] 
compare Tor EE,"THAT 'md LCTfi AT. Table 4,3 is a. listing of 
I>(1) thr-ju^n 0(10), as defined above for the two* with 
E - r p and h - .75 


Move 


LETRi'-.T 

HCTitAT 

l 


.4531 j 

,39063 

2 


.15?53 

,15733 

3 


. 05262 

,06507 

ij 


.01867 

.02735 

5 


.00670 

.01164 

6 


.00243 

.OO^OO 

7 


1 00059 

,00216 

8 


,00033 

.00094 

9 


,0001? 

.00041 

'10 


«00004 

.00013 

SUM 

P 

.68765. 

.66035 

CORRECTION FACTOR 


.00003 

.00014 


and ‘ 1 .68765 -. ,66102 


'Table 4,3; 


Chances of winning under HSTRAT and 
for the first 10 moves* with E "i + 5* 


lstbat 
A ■* ,75, 





The ratio between successive terns for LSTHftT 
begins at . 3"i66l and: increases to *'iYlJti by the JO^ 
move. For rfTTRAT tne ratios r,o up j rom *40^1 to , 4';57j- 
This leads to the fact tnat here LSTKAT f s advantage is 
entirely contained in she first move (For other E and A, 
it is sometimes oont&lnsd in the first two,; Then the 
cumulative effect of cautiousness" begins to pay ofi + The 
fact that the Chance of winning, on the 6th move is twice as 
much under rt'TJh'-.T a.s under LET RAT is dun to the cumulative 
e:foct of n*v:lng used chat strategy on the first b moves * thus 
reducing the chance of winning early in the game, rather than 
to any inherent superiority of H3TRAT once you ore that far 
along in the game and position values are small, This will 
be shown as a byproduct of the work in the next chapter. 

Now turning to strategies with more than one look-ahead, 
we note that in a way 2HSTRAT is an extension of HSTRAT, 

One could easily continue this extension to a 3HSTRAT, etc., 
with each new strategy a slight improvement over the others, 

For instance 7 in Figure ^.1, 2HSTRAT would move to the A 
position* then notice the loss at A^ and move to aB. However 
3H3-TRAT would see the loss riflht away and move to B, which is 
a greater value than AB, 


\.Q 


f 




Figure ^ + 1: A possible frame tree, labeled with the values 
of the points. 

So we have the general concept of XHSTRAT,- where 
X - In the limit we obtain an unlimited look-ahead 

strategy *H?1TRAT whose effectiveness serves as a limit for 
al 1 such strategies, ^HfiTRAT looks down the hlph branch of 
a fork, and its hirr;h branch, if it splits, and then its nij^h 
branch, etc. , until an endpoint is reached,. If it is a win, 
you move to the original high branch. If a loss, move to the 
low branch, (A very inefficient way to use unlimited look- 
ahead., even though it agrees with the One discussed in chapter 
2 on the first 2 or 3 looks* depending on A, Also Inefficient 
due to the repeating of previous lOOk-ahends after each move, 
but this is a, common fault Of all XHfJTRAT's). 
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Similar]y* an «L$TRAT may be defined as a limit of 
Xl'STRAT r s* "Table 4.4 gives the evaluations of these, and the 
already defined two look-ahead strategies* for various E 
and a. 


E 

A 

w 'b 

W SL 

\s i 

W 2H 

W «H 

.5 

♦ 5 

.61315 

.60013 

,61588 

.54025 

, 6l 583 

♦ 5 

. 56667 


.6fc84o 

.68664 

.61385 

,64565 

Li 

. T5 

. 7 ■■ 4 ;.d 

. 7 IS? i 

.73524 

.67301 

.68967 



.84587 

.84783 

.354 6? 

.63360 

J-'i 551 

.5 


. W9T 

* 91575 

* 91640 

.90915 

.40941 

► 5 

.99 

, 9305B 

.9B&63 

.$8076 

. ',8034 

.98040 

.95 

.75 

.54486 

.56354 

•,61653 

* 47841 

,52694 

.5 

.75 

.mis 

.7X673 

.73324 

.67501 

.68967 

.75 

.75 

.7^183 

.780?4 

.78183 

.76470 

,76722 


Table 4.4: Probabilities of winning under BSTfiAT, 2LSTRAT, 
wLSTRAT* 2HSTRAT* and -HSTRftT* over the class of 
gomes of TSI with constant split ratio. 


For A = .5, w 


soil 


w 


iH 


because i>H5TRAT and « 


reduce to the same thing. VL,_ would not equal W™ for 
A ■ + 5 because they would differ on where to move if the 
point looked-ahead to were a split point. 


LSTftAT 

last 
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A much mor* interesting table can be constructed 
by using Tables4,1 and 4.4 to rank nil the strategies as 


to effectiveness: 


E 

A 

1 

3 

3 

.6 

,5 

«»L^-tie- 

—H 

B 

.5 

.66667 

5&L 

B 

2 L 

. 5 

.75 

seL 

2L 

E 

■ !> 

0 

» 


iJ 

M 

B 

p. 

+ ^ 


»li 

?.L 

B 

* 5 

♦ 99 

taL 

2L 

B 


■ :*5 

.75 

■"L 

. 2L 

B 

.5 

.75 

wL 

SL 

B 

.75 

.75 


B 

2L 


4 

5 

6 

7 

2L 

L 

2H 

K 


L 

2H 

K 

mR 

L 

2H 

H 

L 

■H 

2H 

H 

L 

wH 

2H 

H 

L 


2H- 

-"tie' 1 —K 


L 

2H 

H 

«H 

L 

£H 

H 


'tie "--L 

2H 

H 


Table 4,4: Ranking of strategies as to probability of winning 
over the c]ass of games of TSI with constant 
split ratio* 

Table 4.5 speaks for itself. It bears out the con¬ 
clusion that Vi,* > W vu * However it also shows that it is 
not true that in all cases XLSTRAT is the best X look- 
ahead strategy: For E = .5 and A == * 5 and A « *6666? 



The wastefulness of the XH^TRAT's "caution Is 


seen from the fact that one look-ahead LSTRAT is better 
than .2HSTRAT, and even, in certain cases,., better than 
»HSTRAT, 

However, rigorous mathematical .iustifications for such 
conclusions are yet to be discovered, One of the ourcoses of 
the next chapter is to get a start in tnat direction. 

As one last of looking at the results of thin 


chanter, let us note ezactty hen/ much the use of ’look-ahead 
in ordinal look-ahead strategies imnro-aa the chance of 
winning, Table ^,5 reives for various combinations of values 
for E and A the ratios of the probabilities of winning 


under the most effective one and two look-ahead strategics, 
to the probability of winning with no .look-ahead (under STRAT) 


ft 

A 

W 

one look-ahead 
ratio 

two look-eheads 
ratio 


.5 

*33333 

1,6? 

1*86 


, 66667 

♦ 5 

1,26 

1* 33 

.5 

.75 

,6 

1*15 

1* 19 

.5 

,9 

.B1S18 

1*02 

1.035 

■ 5 

-95 

.90%7S 

1*005 

1.01 

; 5 

.99 

* 9&020 

1,000 

1*000 

5 

-75 

*4&?30 

1*21 

1. 32 

*5 


.6 

1.15 

1.19 

= 75 

,75 

.69231 

1.11 

1.13 

Table 

4,5 Improvement in 

used in ordinal 

chance of winning 

strategies over 

due to look-aheads 

the class of games 


with constant split ratio. 


CHAPTER 5 

DEPRECIATION FACTORS AND NON-ORDINAL STRATEGIES 

So far in this report* strategies have been evaluated 
&s to their afreet on the whole njame. In this chapter we 
1 ooK at theft in their effect from any riven position, and 
try to do this without detailed projections Of what will 
happen later on t thus Perhaps enablin' - us to decide which 
Took- ahead Et raters t- use, move by move. 

To this end we introduce the concent of depreciation 
factors"' to aid in our evaluation of the relative contribution 
of "caution' 1 and 1 aggressiveness 11 . For the time be-in.;-* we 
will be concerned with the class of TCI yarned* with constant 
split ratio A„ and an addition will concentrate at first on 
the ope look-ahead strategies for simplicity, 

suppose we try to evaluate the probability of winning 
under L5TRAT from a position which is a split point of value 
about which tne only other information available Is the 
value of its branches* which here will be assumed to be deter¬ 
mined by the constant split ratio A< The probability of 
winning; is clearly less than 'A r because we car. under the 
given strategy look only at a limited portion of the game tree 
below the point* and only move down a single path. This has 
already been seen to be true for all the limited look-ahead 
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strategies so tar considered for the cose where X « 1 
and we are at the beginning of the gome. However, sunpose 
We do take X to ho the probability" oj winning in such e. 
caae . We might improve our estimate by addin." the actual . 
chance of winning under Li?TRAT on the first moie fTom the 
friven position, KB3 ■+ (l-EBX)EAX, to the chance of movir.fr 
to a split point, (l-EBX)S, times the ^lue of that point, 

AX, which is being used ns the estimate of the chance of 
winning from this second sol it point. The resulting formula, 

call it V L , is 

V- * EEX + (l-EBX)EAX + {1-1BX)S(AX) 

i..C 

= EBX +■ (1-EBX)(AX) = EBX + & - E/iBX* 

Evaluating HSTRAT in the same way, we get 

V„ * EAX + SAX + E(1-AX)(EBX+SBX) 
li 

* AX + E(1-AX)BX - EBX + A3C + EABX 2 

Our over-estimation of the chance of winning from a 

split point has given rise to the interesting result that 

V = V . However, rather than puzzle over the exact reason 
L H 

it works out this way, let us try to correct that over¬ 
eat imation. It would be very convenient if the actual chance 



of winning fr*ra a split point was a constant, depending only 
on the e tratefjy Peine needs times the value of the point. 

Such a constant could be called a depreciation factor . 
However., it is really too- much to assume (in T^X) tnat cuch 
a. constant e'/.lsts* independent of the value of the position* 

>Jhat is actually needed is probably a '’depreciation 
function" of E h A* and X. However* we can Ret an estimate 
or this function by assuming that it _is constant, at least 
constsnt from one move to the next* Xhpn letting K stand 
for the constant* KX would bo equal to the estimate of 
Probability of winning which is calculated analogously to the 
calculations of V_ and 

KX = EBX + (1-EBX) [EAX -h SKAX) 

Solving for Ke 

# 

v - %W* + EXA - E^ABX ? E( l-B ABX) 

f 'L v . 177“ 1 - SX + ESABX 

X + E5A.BX - okX 

As X approraches O* (actually a function of E, A* 

and X) increases to a limit Of E/{l-£A) 

For HSTRATj we would have 

KX = EAX + SKAX + E{1-AX>[EBX + SKBX) 

_ EAX + E^BX -_E £ ABX g E(a+EB) - B^ABX 
H X - SAX -ESBX +ESABX 2 1 ‘ ^Ta+SB) "+ ^S75a 
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As & approaches 0, K, f increases to a limit of 


1 - S (A+ EB } 

Interest inrljr enough* the limit inn values for K H 
and K l arc YJ^ and W L for TSD {Chapter 3)- This 
surprise hat a f^ood explanation, however, Ae X approaches 
0* the independent name TSI ^et closer to the dependent 
TED (the (1—Xj iactors you divide by when you discover 
losing endpoints, approach L.) And in T5D and are 

the depreciation factors, which in fact arc constant through¬ 
out the ysme, as can be seen if one j^ces o -cr the analysis in 
chapter >, Because oi this constancy, they f^ive the actual 
probability of winning when multiplied by the va 1 ue of the 
position. The top position has va^ue 1, so the chance of 
winning the r^amo is oimnly the depreciation: factor itself. 

Now, if for i^iven E and A we superscript the K r s 

to let k 7 = limit K? , we have, from Theorem 3,3 
h x-*o L 

B _ v m ^ ^ EfA+EB) 

1-SA K h - K ij " 1 - 'SXA+EBJ 


It can be concluded from this that 



E - E^ABX _ E(A+EB} - E S ABX V X 

1“ 5A + ESABX - 1 - S (A+EB) + E5ABX H 


.because exactly the same terms are subtracted from each 
numerator* and exactly the same terms are added to each 
denominator, ■ J ’ ■ - - 
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Nowj since in T$I K* is not constant for all 

y 

position values X, X K T does not give a precise estimate 
of the probability of winning from position X under 
LflTfSATj in fact tne real probability viill be eomewriere 

y „ 

between X KV end X K T , It is intuitively plausible from 
the above inequalities however* that the chance of winning 
from y. under LET RAT will be greater then that under 

no .matter what value K. This is in support of the 
claim movie in the ia.sL chapter that the fact that under 
HETRAT there is a greater chance or winning on the c* 1 * 1 
move than under 1.3TRAT is only indicative of the cumulative 
effect, and not of any real superiority of HSTRAT at that 


point In the game. 

We can further explain the kind Of results summarized 
last chapter in table ^,P as follows. Suppose you are at a 
position of value X small enough so that the limiting 
formulas K? and lC approximately hold. Then the probabi- 
lities of winning are^ respectively 


XE 

1 - SA 


and 


XEfA+EBj 


Under LSTRAT the chance of winning on the next move, if K 
is small enough* is approximately EaX + EBX = EX, so the 
formula can be interpreted as representing the sum of the 
chances of winning on each successive move, the first term 
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being JCE a nd tne ratio between succeeding ternia being is A. 
(This was the viay we got the formula in the case of TSJJ in 
chapter Y). Similarly p for HSTRAT the first term is XKfA+GB) 
and the ratio S{A+EB). The ratio for HSTRAT is greater 
than that for LSTRATi but this is outweighed by the Prepon¬ 
derance of the first term In the LETKAT series. 

If this analysis is correctj the ratio D(l) •'D(l+1) 
described in chanter 4 should approach the ratios described 
in the last paragraph. This is shown In table 5,-l below 


A HSTRAT: ^^ S{A+EB) LSTfiAT: 5 A 


* .J ‘ 

5 

.37460 

♦ 37500 

. ?4 [#2 

.25000 

, 5 

66667 

+ 4l6l5 

.41667 

.33^65 

* 33333 

.5 . 

75 

♦43573 

,43750 

.37118 

.37500 

.5 . 

9 

. 46774 

.47500 

.44318 

.45000 

.75 ■ 

5 

.21867 

,21875 

.12486 

.12500 

♦ 75 ♦ 

75 

.23341 

.23437 

.18464 

.I 875 O 

Table 

5 . 1 : 

Comparison 
of winning 

between actual 
on 9**^ and 10 

ratios between chances 

^ moves With theoretical 



limits, for 

-HSTRAT and 

LoTR AT 

in games of TSI 


with constant split ratio* 

With the introduction of depreciation factors, it 
becomes possible to discuss intelligently non-ordinal strategic 






which make use of the actual values of the branches of a fork* 

not Just their relative ranking To do this we first drop 

the assumption th^-t the split ratio is constant, and then add the 
the 

assumption that depreciation functions Just derived for 
constant shl.it ratio A also hold approximately for a 
distribution of split ratios merely with expects! value k. 

The advantage of non-ordinal strategies* if there is 
any, will come from the possibility that, over the class of 
game trees, the possibility of winning from a split point 
whose branches’ values arc known may -'ary a little depending 
on how the top .-'alue is split between its successors. Thus, 
given two positions which are split points ah! whose split 
ratios are known., it may be possible that even though 
position X has value higher than position Y, the chance of 
winning from position X is lower, because its value splits 
in »p unfavorable way. 

The ordinal strategies to which such considerations 
might apply are HSTRgT, 2H5TRAT, and B5TRAT d for in all 
these we ordinarily can move to a split point whose cpl.it 
ratio we have already seen in loo-k-ahead. The use of this 
information to modify the strategy in the cases of BSTRaT 
and HSTRAT will be shown. The part of the strategies which 
tells where to look will remain the same, but the rules about 
where to move, given the results of the look-ahead, will be 
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altered. There is alec the possibility of strategies with 
more than one look-e-neAd, where the results of the first 
look-aheads (the exact values of the branches seen) nay be 
used to determine where to look next, Such a atrate^ is 
the unlimited look-ahead strategy discussed in chapter 2 r 
where that position with the highest alue of all those you 
have not yet seen* but Know the value of, is looked at next. 
However, this is the onJy example of such non-ordinal 
strategies studied so far, 

One unfortunate aspect of the non-urdlnaJ strategies 
we do study is that they do not lend themselves to simple 
evaluation techniques. This is because an expected value of 
the split ratio* like the A v/e used for ordinal strategies, 
is useless when we want to utilize the fact that the .split 
ratio varies according to a probability distribution. 

Evaluation will have to be done by computer using the Monte 
Carlo methods whose reliability was brought into question in 
the last chapter. However if we make consistent use of the 
random number generator* we should be able to get an estimate 
of the amount of improvement non-ordinal information allows us. 
(and a]so its cost in computer time). 

So* to define these non-ordinal, modifications; Under 
BSTftAT the depreciation factor is found a.s before 
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K7, - EAX + (l-EAXjEEX 4 K(l-AXjSKUi{ i- (1-KDXjSKAX 


K 


EX- 






X ‘EoBX4- EE mBX- - 3 AX+ E3 AB X 


E-E ? ABX 

17 sTa+ 61 TR?E 5 JM 


Limit = K™ - E/( 1 —f a+BEj ) j which combines the best of 

X-K> 

both HSTRAT and L3TRAT -- the hip;h chance of’ winning on 
the move, and the hirth ratio between such chances on succeed¬ 
ing moves. But this was to be expected, fit least if! the 
limit. 


By using the depreciation "factor 1 ', we can ret an 
idea of the probability of winning from the ton of a. fork 
wnose EDlit ratio is Known. Define 


k b (e,a*x) 


(E-E ? ABX)X 

r=“iTA+W)' + 2'ES^X 


as a depreciation “function 11, giving the depreciated value of 
a position with value X. Then let our known fork have top 
value X and split ratio R. The chance of winning will 

be approximated by 


C b (E,A,X,R) ^ ERX + (l-ERX}E(l-R)X + E(1-RX)S K^fE,A,(l-R)X) 

+ (1-E(1-R)X)S K b (E,A,RX) 

(Note that we are still using, A as the expected value of the 
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split ratios we don’t Know shout yet,) BSTRAT would be 
modified only in the case when both high and low branches 
of your position have been found to be split points. Denote 
the value of the low branch by L, and the high branch by H, 
end suppose that their split ratios are R,, and R„ . r ?7e 

—-- n L 

then compare C b {E,A,H,I h ) and CgfE, A,L,K l ) and move to 
the branch with the highest value for this. In all probability 
this would be the H branch anyway, but sample computer 
calculations show that if H/(H+I/j < ,55 there is at least a 
possibility, correspondingly greater a^j the ratio approaches 
, 50 , that the I. branch may be chosen, 

Monte Carlo methods, implemented as described in 
chapter 4* yielded the following evaluation of BSTRAT and 
its modified version, as to chance of winning. In 10,000 
games, BETRAT won ,72435 of the time, with a variation 
over repeated trials of only ,00010, 'The modified version 
B'STRAT won .73200, with a variation of .00030 over 
repeated trials. This is an improvement of approximately 
,00765- This is a considerable amount when you consider that, 
■in Table 4.3 for E - , 5 , A = .75j - W B is Only .Q0235j 

and so this modification probably maJces BETRftT the most 
effective two look-ahead strategy we have. This is born out 
by a Monte Carlo evaluation of 2LSTRAT which yielded an 
estimate of at ,72810 + .00020, clearly less than the 

estimate' for ‘ Yl^, * . „ ' : ~ r ; r - _ - 
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However B'iEjTRAT* though no re effective, is no re 
costly than BSTRaT in the sense that it took slightly more 
than twice as much tine to play 10,000 rtomeSj due to the 
extra calculations involved in B’STRAT, This should 
probably not be talker, to Seriously, since in most real, panes 
the tine involved in such calculations would be much more 
seriously over-Weighed by the time needed Lo simply generate 
the position ■■aloes, and anyway* we could have skipped them 
sxceot when Fi/fitHj f . 55, which is the only tine they could 
be useful + 

Note that this non ordinal strategy is cuite sophis¬ 
ticated in comparison to what nipht have been suggested 
without the -iid of depreciation factors, for instance, 
confronted with a situation such as in Figure 5.1 t one mipht 
■think to po to the lower branch because its high branch has 
the highest value of all points at its level. 


l.o 



V inure 5.1 * Example of part of a pame tree for T^I 
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However* since ,6 ■'{.^+*6 ) > *55p this would probably be 
wro-htf assuming that our previous calculations are Indeed 
.snpl icnbl e*. and applying such a simple rule over the lonf 


run would surely be contra-productive. 

Turn inn; to HoTRAT*, a.^nin we must pass up simple non- 
ord inn'll modi fie at jons, for instance t after looking ahead to 
the ,6 point* deciding to no to the .4 because is 

greater than „3. Instead* uhinn the depredation factor 
derived above, we del’ine 



and, with R and T ns alone 



Suppose we are at a fork whose bottom Points are again H and 
L, and we find by look-ahead that the H branch splits* wdh 
split ratio R. Then we car, compare d^E, A*H*R) to 
EL + £K h ( 5*A*L) and move to the H branch only if the first 
vi ue is the greater. This modification was evaluated by the 
Monte Carlo methods already described* with SK h £e*A*L) 
replaced by SC^(E*A,L*Aj, which should be improvement* although 
pa proof has been found. The resulting fraction of wins was 
, 67^65 compared to . 67^70 as was obtained before for RSTRAT 
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by this method, an increase of about , OOi : QO> and nowhere 
near enough to make it better than L5TRAT, which has an 
Advantage of about . G2&50 over tlSTRAT. The tost was ar*aln 
a doubling: of computer time for 10,000 Barnes, hut, likewise, 
wain wou 1 ri have been reduced if, in the cases where 
was too rreat, the calculations of 0 had been omitted. 

Under this strategy the cut-off point would be about *5^ 
for \ ar. ,r e (in absolute terms) values of J3, decreasing to 
perhaps * ^6 for H c ,00-6, These cut off points were 
determined by computer evaluations o ? C for various H and 
L, and why it should be what it is, and why it should riencnd 
on the vglue of H itself, are questions open to further 
research. 

The results of this research into non-ordinal strategies 
is summarised in Table 5*£ below 


Strategy Fraction of wins 


Fraction of wins 
when modified 


Differenee 


B2TRAT 

HSTRAT 


.7^35 

,67070 


. 7‘1200 
.67265 


.00765 
,00195 


Table 5 +it Comparison of Monte-Carlo Results for Ordinal 
and Non-Ordinal Strategies In Games of TSl 
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CHAFTE.fi 6 

VARIANTS OH THEE SOLITAIRE 
AS A SUGGESTION FOR FURTHER STUDY 

The obvious first surest ion for extending the results 
so far reported wou?d be to study in more detail the ordinal 
Strategies in True GoJltaire with more than two look- ahead 3 ,' 
or perhaps strategies which are not 'limited as to number oi 
look-aheads but 45 to the number of ply deep in the gaine tree 
they may look. 

Going beyond, the specific limit at ion to the rules of 
Tree Solitaire, the obvious second suggest! on for extending 
the results would be to stop the restriction to binary game 
trees, and allowing for the possibility of more than two moves 
from a given position. We could consider the ease of ternary, 
aryi etc, trees, or we could even let the number of branches 
at a given position vary according to some probability 
distribution, A kind of tree solitaire could bo defined on 
such trees, in our strategy definitions "'lovr 1 could be 
replaced by "second highest' , and evaluation could precede in 
perhaps much the same Way, 

However, rather than leap to such attempts in this 
last chapter, we shall continue our limitation to the binary 
trees for which our terminology has been developed, and 
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consider some variants of tree solitaire preserving this 
characteristic. 

The principle failing of Tree Solitaire as a model 
for the kind of games involving look-ahead which we usually 
encounter, apoears to be the fact that the values of the 
nositions naturally decrease as you get further on in the 
game, This is the main reason we had to have "depreciation" 
factors and the probable explanation of why aggressiva 
i/lTkAT did bettor than cautions HSIHhl. How can we correct 
tnis failing? 

There are two kinds of modification, which can bo 
effected independently. One is to chanRe the game tree and 
its generation, the other to change the ,f valuation M of the 
tree, the method of assigning values to its points♦ We would 
like to keep the basic approach of generating the tree first. 
Independently of the values which may later be assigned to 
it, although these values may themselves reflect tree structure 
"as is often the case in real parties and our evaluations of 
positions in them. 

An example of modifying the tree would be to have E, 
the probability that a point is an endpoint, be a function of 
its level in the tree [how many moves deep in the home it is) + 
This would probably not complicate matters much, and indeed, 
the diagram-comouter program methods of chanter 4 might still 


be useable* In ad flit Ion the essentia.! feature of each point 


being judged as E or an S independently of ail the other 
noints Is retained, However, by itseJf\ this modification 
does not solve the problem referred to above, and this 
chapter will now further restrict Itself to only those 
modifications which alter the valcat j o n of the game tree, 
leaving this suggestion as , H ust that -- a suggestion, 

Three Dossil ? 1 e modified games wil 1 be discussed, ?n 
a sc and L ny. order of usefulness, 

\. Given "ome probability distribution of numbers 
between f? and i, riasign v&luoo to the endpoints of the 
(finite) tree. The value here represents iho probability 
that the endpoint is * win. For every Sol it point of the 
tree, let its' value be calculated from the values c : the set 
of endnoints which arc its successors* immediately or farther 
down In the tree, end let It be the probability that at least 
one of these Is a win, I.e.* the chance that there Is a win, 
beneath that split point In the tree. If the act of successor 
endpoints has 3 numbers, with values k, B, and C, then the 
value of the point would be 'A + B+ C-*AB-AC«BC + ABC. 

This does not solve the problem raised above, for the 
values of points decrease as you fto down the tree (although 
not Indefinitely, as they can in TS), but it has one interest^ 
Ing facet* Consider the problem raised In chapter 2 of finding 
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the least costly unlimited look-ahead strategy which p;uhjrhntees 
a win (if there are any wine In the “nme), The answer there 
for Tree Solitaire was to look, first at the point with the 
highest value. In Game 1, if the distribution of ondnoint 
values is such that the values are all equal to some constant 
A, the least costly strategy, in terms of number oS positions 
7 ^ou need to look at, is to look at the lowest valued point 
avail ab Jd* 

Proofs From the value of n on lit point, wc can, knowinr A* 
calculate the nu.mber or endpoints beneath it. The lower the 
value the fewer the endpoints. However, the lower the number 
Of endpoints beneath a point in the tree, th^ higher the 
proportion of such endpoints to the total number of points 
beneath, that point. (If K is the number of endpoints, there 
are K-l split points, Count inn; the top one. K/2K-l decreases 
as K increases). Since all endpoints have equal probability 
of beinp; a win* the least costly look-ahead procedure will be 
the one which wastes the least time in Getting x OQ ^ a ,t 
endpoints, This is accomplished by looking at the lowest 
point, which can be expected, by the above consideration, to 
be the closest one to an endpoint* Q, E. H* 

The reason this works is that the value of a point is 
not independent of the structure underneath it in the tree, is 
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In 1 act a function of the number 0 f - endpoints* Even If the 
values of the endpoints were not equal but were assigned 
according to-a probability distribution as explained above, 
there would still be a dependence, from which some information 
about the tree structure could be obtained. Such an arrruirre- 
ment is probably not at all unreasonable since such Informa¬ 
tion is probably available in the valuations used in actual 
names, however, it is an exenrole of the non standard coding 
nnenomcna and complicates matters unduly (and arbitrarily yin 
jj theoretic treatment even though it is valuable in giving 
US an idea of hovj our expectations may _gq wrong in such 
s'tuat50ns- Herb the best limited loosi-ahhed strategy would 
probably have to make use of such information., and. the 
evaluation of any strategy would have to take into account 
the dependency. An ability to count binary trees with a given 
number of endpoints might be needed, and there is no easy 
Closed formula for that, 

2. Similar problems arise in this second game, due 
again to the fact that values are assigned from the bottom of 
the tree up rather than from the top down, and thus can reflect 
tree structure* 

Assign values to the endpoints as in. game 1* However 
this time let each split point have value equal to the 
average of the values of its immediate successors. We now 
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have avoided the orob 1 era of having natural Xy doc re as inn 
position values. However, again problems in evaluating 
stratarj.es will .arise because the value of a position does 
teHl us a little about the tree structure beneath it. 

Although the ejected ,|f a ; ue or the value or a split point la 
tne sane as that of an endpoint, ^ts variance, because it is 
an. average, will be smaller., as a function of the number of . 
end-points beneath it. Therefore a point with a vaiue very 
Jar i'rom the me am is llhely to have only a few endpoints 
beneath it, or be one itself. 

This game has beer, mentioned not so muen for its own 
importance, but because it suggests an idea for a game where 
v- lues for positions are independent of the tree structure 
beneath them, and yet are '"likely" to be the average of their 
successors. 

3. Decide on a probability distribution 3 with mean 0* 
To the top of the tree assign a number T between G and 1. 
Given a fork in the tree whose top value C has already beep 
assigned, generate independently two numbers A and B 
according to the distribution. Then the values of the branches 
are C'+ A and C + B (i.e., the values of the branches are 
Chosen according to the probability distribution 3 shifted over 
so that C is its mean). If a point Is an endpoint with 
value P, the probability of its being a win' is P if 


and 0 if P < 0 


0 £ P < 1 , 1 If P > 1. 

The valuation in this game is an idealization of 
the way which computer position evaluation functions might 
toe thought to wori; with regard to successive positions, end 
In that sense this game should toe a much totter approxima¬ 
tions to tne reaj vantes game playing urograms are written to 
doe.i with* With this hope, the rest of this chapter is 
devoted to show!nr how the idea? and techniques of the 
earlier chapters can toe used as a start on the evaluation of 
strategies in mantes of form 3 . 

To construct a diagram that might toe a do r opr i ate, vre 
note that although the expected value oi either branch n 1 " a 
forh. is the value C of the top point, the expected value of 
the higher branch is somewhat greater than G and of the 
lower branch, somewhat less. Let us suppose that ^ is 
symmetric about its mean; then the expected value of the 
branches would be G +■ A and C - A, where A is determined 
from h. This yields a diagram as in Figure 6.1 based on the 
assumption that the expected position values are always realized* 
The values on the lines can be filled in according to the 
ptr&teijy you are considering, and. the computer programs of 
chapter 4 could easily be modified to evaluate all the 
strategies that were evaluated for TS* 
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Figure 6.1: expected position values ani possible moves In 
Game H. 


However, the reasonableness of the estimates of probability 
a! winninn thus obtained would denend on how the actu&l 
distribution E scattered its values and what effect this 
rni^ht nave. As in the case of Tree Solitaire* it's hard to 
tell whether usin^ the expected values of Position values to 
construct a diagram leaves out any essential consideration ^ } 
and here we cannot fall back on the restriction to constant 
spilt ratio* The effect could be checked by Monte Carlo 
Method s. 

The results of sample hand calculations for T - 1/2, 
A = 1/6 are summarized in Table 6,1, where after move 1 the 
formulas were too complicated to evaluate. 
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Move 

ProbAbility 
HSTRAT 

of winning 

LSTR AT 

. 1 . . 

.361 

, 444 

? 

. ^4? 

t ?;4 

3 

* 14 9 

. 104 

sum 

.755 

,7ft? 


Table 6* 1: Hsni calculations of chances of winning under 

HSTRAT LSTrlAT over the clssn of r;ame:i of 

form 3. 


It Is annaront that even though I3TRAT 3 s ahead at the end 
o? r j riKjvas, ’ri'JTitAT will predominate when the sum is taken 
over, oay,. 10 movesj by which time the F^ame should-be over. 

However * a more exact estimate can be obtained by 
caleulat.i h^ the probability of losing, for tha-t sum turns out 
to converge quite rapidly* See Table 6.2 


Move 


Probability of Losing 
H3TRAT LBTRAT . STRAT 


1 

£ 

3 


,0556 
. 0191 
*0069 


SUM ‘ .0316 

Sum +■ correction factor.OB50 

1-5 [TM — ^ 1 ^ ^ c ji k 

I - bU.M of winnlnft ■ 


*139 
.035 
, 000 

- 174 

*m 

*e ?6 


. 167 
.04? 
.000 

,209 

.174 

.791 


Table 6.2- flsnd calculation oi Chances of losing under HSTRAT 
l^TRATf and 5TRAT over the class of games of form 3 







STRAT was thrown in for "ood measure to show the improvement 
due to look-ahead over the best one can do without any. The 
third term for JjSTRaT end 3TRAT ia___0 .because with A 
and h: as they arc, from the "^rd mo r n on under these strate¬ 
gies yo'j can ro only to positions with -/afue greater then or 
equal to T. The conclusion one draws from this table is that 
HSTRAT is the best one-look-ahead strategy, It is reasonable 
to think that this ^er.eralitea to all Allowable values of A 
and E, How that values of joints no longer need decrease 
later on in the ^ame there is a premium on "caution" and 
"aggression 1 becomes "recklessness' 1 . 

Although a proof of KSTRAT 1 s general superiority is 
not yet, known, it can be shown for the limitinr; case where 
A - 0, i + e, all positions have the same value T. (This is 
the same as game 2 with all endpoints assigned the same value,} 

THEOREM 6.1; Let represent the chance of winning under 

HETRATm and W L the chance under LSTRAT, over the class of all 
.Fames of form 3, in which every point of the seme tree has the 
same value Tj 0 < T < 1. Then W„ > W. for all E (the 
probability that e. point is an endpoint) 0 £ E < 1. 

Proof: Under HSTRAT the thence of winning at any particular 
position is 
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KT -i- e(l-T)KT - ET{ 1 + E - ET] 


The chance of contj nuin/* on Jn the frame is 

o + Efl-IJfj ■= o[l +■ K - ET] 

Kffl+E-Frrj tfr(li-E-ETj T(UE-I2T) 

u0i J,J Ji ' L-aTi i-’S^et] ” i-(T-J5}(i*-s-erT " t-te+e 

For L3TKAT the chance of winning at any nosition is 

ET +■ (i-ET)ET 

atv] the chence of continuing in the frame is 
(1 - ETjS = 1 - E - ET + .E^T 

„ ET(^ST) T(2-ET) 

" E(1+T-ETJ “ (1+T-irr) 




Tfl+E-ET) 

T-TE+E 


Tfg-STj 

1+T-ET 


This will be >0 if 


(1+E-ET)(1+T-ETJ - f2-ET)(T+E-ET) > O 
This can be simplified to 

1+T-ET+E+ET-E^T-ET- ET^+E ? T ? -£T-2E+2ET+ET 3 +E^T-E ? T ? 
= 1-T-E+TE" {1-E)(1-T) > O 
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(because of the bounds on K and T ) 


q. K.B. 


h L or A f treater than 0 and on sum i nv, the /'ame works 
like diai^rejut 6*1, it can be shown that both the ch&nce 04 
winhin?; on the move and the chance of continuing past it are 
Improved* for both HSTRAT and L3TRAT, but it is not irnme- 
dln.tely apparent wnieh vains the most. It is at least plausible 
to believe that HSTRAT maintains its superiority* Perhaps 
somethin 1 ' depreciation e actors will be inventable 

{although here they n:"ht be *' annrcc lotion factors' 1 ^ an'i can 
be used t.o solve this problem, and then to ^o on to propose 
non-ordinal strategies. 

At any rnte> the avenues for t urther study are wide 
onen T and hold prospects of more and varied results. 
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APPENDIX A; Programs lor evaluating strategies in chanter |J 

CALLERt This is the aa]linF, nrogran for the rest of the 
programs* on CT5S+ 

OUTPUT: This KAD external function is used by an the 

following strategy functions to sum find print their 
their result so the values 0(1) for I ■ 1 throuj^h 
10 representing the chances of winning on each of 
the first 10 moves, and the ratio between succeeding 
terns. The sum of the D(I) is Increased by r 
cor ree t ion f ac 1 0 r ar r i ved at toy as sum in g that the 
ratio between successive terns, from D(9) tin, is ■ 
a constant, CORR = D(lO}/(1. -D(9)/D(10)) - D(10} 

I r. '■ no fo 1 low 1 ni • AD external functi on s ;mi c h cnlcul ate 

D:1 for the various strategics discussed in chanter 4 when 

presented with values for E end A, V(I) represents the 

value of the 1 th point from the left in the corresponding 

diagram* at the level under consideration. T(Tj represents 

the probability of getting to a position with such a value at 

that level h All the programs start and end with the same 

statements, which are 

EXTERNAL FUNCTION 

PROOF AM COMMON V,T,D,A,RjS,^, 

DIMEWSIO U Vf10j,D(10),T(10) 

INTEGER 11,13.13,14 


FUNCTION RETURN 
END OP FUNCTION 

and so only the differing parts will toe listed. For each 
strategy the diagram will be presented, comments If necessary, 
and then the program listing. 
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lstrat 



ENTRY TO LSTRAT. 

V(l) « £*A 
V(2 - E*B 

T(l - l.-V(S) 

D[1) - V(l}*Tfl)+V(2) 

THROUGH fa, FOP J-?pX*J,G.10 
V(2) = 7 {l) * B 
Vfl) = vfaW 
T f 3 i “ T (] i * 5 

t i - 

Ml ]>(J,j = TflFV(l)+T(2) # Vj2) 
PRINT COMMENT $ LSTRAT* 
EXECUTE OUTPUT. 



70 


H ST RAT 



ENTRY TO H3TRAT. 

V fO 1 » 0 
V(1 - A 

VfS) ±* B 
T(0 = 0. 

T 1 \ = 1 

t|b = eWi.-a) 

Dfl) = E*(T{1)*V{1)+T(3}*V(2)) 
THROUGH Hi* FOR J = S^J^IG 
V(J+l) = V(J) # B 

THROUGH M2, FOR L = J*-l,L.LE.O 
M2 VfL) * V(L)*A 

T(J+1) ^ E*S*(l.-V(j))rr£J) 

THROUGH M3, FOR L - J,-l,L.LE.O 
M3 T(L) = £*T(L)+E*S*{1.-V(L-1))*T(L-1) 

= o. 

THROUGH HI* FOR L » 1,1,L.G*J+1 
HI D(J) - DfJHV(L) # T(L^H 
PRII'-fT COMMENT $ H£TR£.T$ 

EXECUTE OUTPUT. 
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2LS1RAT 


® 1 





The lir.es arc not connected at the AB and P, B points to 
emphasise the fact that in TOIj under this strategy, you 
would not move from B to ^"iJB and then to A r B etc, 7ou 
move E to aB only when you have seen by previous look- 
ahoad that aB a win. otherwise the program is quite 
similar to that tor LSTHAT 

ENTRY TO L3TRAT 
V(l) - E*A 
VfS) - E*E- 

Tfl) = l.-E*B*{l.-S*A) 

T(2- = 1. 

I 1 (3) = 1. 

0(1) - T[1)*V(1)+T(2)*V(2) 

THROUGH Mlj FOR J = 2*1^0.10 
V{2) - Tfl)*E 
yn > = y f 1 } * ^ 

Tt 2 ] - Tf1)*S+T(^)*S 
T{3) - T(i)*S 

Tfl) * T(l)*S*(L-V(2)-V(2)*S*A) 

MI O [ J J * T(l)*V(l )+T(£)*V(2) 

PRXI/P COMMENT $ ?LSTRAT$ 

EXECUTE OUTPUT^ 
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2H5TRAT 



*The dia^ran does not work like the ones before. Suppose you 
are at AS and it is an s. If you came from B then you 
Know that is a split (else at 3 you would have moved’ 

to end so the chance of moving from AB to AB^, t^iven 

that you came from B is S^Efl-A-B). However, if you Cane 
from At this chance is 5(1-A5B) + SE(l-A J B), Therefore 
you cannot associate to each point in the diap.ra.Ti a single 
Value representing the chance of netting to a point with that 
Value at that level, but need two numbers. One for each Way 
of getting there* and you must keep track of them separately. 
In the urogram R[l) is the possibility of coming from the 
upper left* T(l) from the upper right. 

DIMENSION H(10) ' 

ENTRY TO HSTRAT 

R(0) ^ 0, 

R(l * 0. 

REE = E*(l* ~A)+S*E*(1* -A*A) 

t[o = 0. 

Tell = 1. 

T(E] = 0.' 
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M2 




Ml 


vfo) = 0 . 

V f 11 A 
V? 2 j » E 

0(1 J = E*(V(l)+R(2)*V(2)j 
THROUGH Ml, FOR J - ?‘ p l, J.'GllO' 
V(J+X) * V(J)*B 

THROUGH MS, FOR L - Lii.0. 


V(L) - V(Lj*A 

R(J+1) - )-*5 

TfJ+ly * 0 H 

THROUGH M3. FOR L J,-1,L.LE.G 
T(l) S*(T(L)+R(L)) . 

r{l> ■ T(L-1 J*S*3*E*(1,-v(l-i)*a) 

1 !-R[L*] }*E*S*((l,-VfL-l ))+S*(l.-V(L-lJ )f A; j 

y(j) - o. 

THROUGH Ml, FOR L * l,I,L.ti t J*l . 


U{J) = D(Jj +V(L) *E* [T(L)+R(L}) 
PRINT COMMENT $ 2HSTRAT$ 


EXECUTE OUTPUT. 
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LIMITS 



ENTRY tO LIMITS. 

G m 1,/(l +-S*A) 

D(l} E*A*G 
T(l) - 1,-Dfl) 

V(l) * E 

THROUGH ML* FOR J ■ 10 

V(l) - V(l}*B 

C - V(1)*G*A 

DfJ) * T(I) # (C*S+V(1)} 

T(l) = 

PRINT COMMENT t HLlMlTi 
execute Output* 



THROUGH M2, FOR J *= 1,1,J.G.10 
C - V(l)*B*G 

D(J) - T(l)*(C+(l*-C)*V(1)*A) 

vi) - vjl *A 

M2 T(l) * 

PRINT COMMENT $ LLIMIT$ 
EXECUTE OUTPUT, 
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BSTRflT 



Ir,ic di fiuj?>T\ iz 'jced a bit differently from the rest. At each 
ooint ^he enanec: of winning, not at tna.t point..* but at its two 
succescors if trey exist, is computed. If the volute of the 
point is 7. r the formula for this is EXE i (l-EXB)EXA. Thus 
the numbers on the lines indicate the probability of (setting 
to the points and finding that they split. This modification 
is necessary because of the way BSTRAT works, and. is related 
to the modifications, for 2H5TRAT & 2LSTRAT. 

ENTRY TO BSTRAT 
D( 1) = E*B*(1-E*B)*E*A 
y Tfoj * 0+ 

Til) - $*{1.-E*B) 

T h) = S*E*(1--A) 

Wo) = 0. 

V(l = A 
V(£) = B 

THROUGH Ml, FOR J - JO 

DfJj = O. 

THROUGH M2, FOP L = 

M? D(J} - D{J)+E*V(L) r T[L)*(B+(l.*E«*/tLJ ) *A) 
WHENEVER J.E. 10, TRANSFER TO Ml 
V(J+1) - V(J)*B 

THROUGH M3, FOR L = J,-1,L,LB,0 
M3 V(L) = V(L)*A 

T(Jtl) = T(J)*E*S*( 1 *-V(J)) 

THROUGH M4* FOR L = J,-l f L*LE.O 
M-4 T(L) = S*({1.-E*V(L] )*T(L}+T£ti-l)*E*(l,-v(L-l))) 

Hi CONTINUE 

PRINT COMMENT $ BSTRAT$ 

EXECUTE OUTPUT. 
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APPENDIX B: Sample Monte Carlo Evaluation Program 


The following MAO program was used, on a CTSS system 
to' evaluate - HETRAT with'non-ordinal “ni&liif £0 at-Ohs,' over the 
class of TSD crarr.es, by playing through a series of randomly 
generated games using that strategy. 


INTEGER I.J 
HRAD FORMAT F*J 
VECTOR VALUES F=$I3*& 

THROUGH Ml t FOR I»l t l,I P G.J 
Ml ' A*3AMMO. fit i 

READ FORMAT FI,E h NGAME 
VECTOR VALUES F1-4F5* 5, F5.0*± 

Cr--0, 

W-O. 

T-0. 

TW=0, 

Hj WHENEVER 0.OR.NGAME, TRANSFER TO END 

*1 = 1 . 

■H4 A-RAWHO* (X)/£.+, 5 
T*T+A 
TN*TN+1* 
m XS-Xl^A 
X>X1-XP 
An. RAH NO. (X) 

WHENEVER A.G.E, TRANSFER TO M5 
JWTANNG.(X) 

WHEREVER £<G.X2» TRANSFER TO M6 
WIN W=4I+1, 

LOSE G=G+1* 

TRANSFER TO M3 
M5 A-RANNC. (X)/a*+.5 
T=T+A 
TK=TN+1. 

H=E*X3+(1.-E)*C t (X3 Jt 75)-C,(X3,A) 
WHENEVER H.G.O, s TRANSFER TO M6 
X1=X2 

TRANSFER TO M2 
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M6 A=RANNQ, (X) 

WHENEVER A.G.E, TRANSFER TO MIO 
A=RANNO»(X) 

WHENEVER A.G.X3, TRANSFER TO LOSE 

TRANSFER TO WIN _ 

MIO XI-XA 

TRANSFER TQ 
END SCORE*W/C 
AVEm-T/TN 

PRINT FORMAT F0, SCORE, AVE 
VECTOR VALUES F3-$2F8>5*$ 

EXECUTE EXIT. 

INTERNAL FUNCTION K,(¥)=Y*( E *(.75+E* H 25}-E*E* h l875*Y 

1 (1 + -5*M5+-E**25)+E*S*. 18? 5 *Y} 

INTERNAL FUNCTION C, f U,R}=E 1, U*R+S*K. (U*R)+E* [1- -U*B) 
1 *(E*U*{l l -R)+S*K l (U*[l,-Rj)) 

END OF PROGRAM 
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